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Abstract 

A wide variety of research is being completed and published in geography education. The purpose of 
this article is to provide a general overview of the different types of methodologies, research designs, 
and techniques used by geography education researchers. Analyzing three geography education 
journals, we found 191 research articles published about geography education, of which, 55 used a 
quantitative design, 65 used a qualitative design, and 71 used mixed methods. This article highlights 
specific articles to provide examples of how a variety of quantitative, qualitative, and mixed methods 
techniques were conducted and presented in the literature. The implications of this article is to provide 
geography education researchers with guidance when reporting research results in an effort to promote 
replicability and longitudinal research. Geography education benefits from all types of methodological 
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designs and techniques, each serving a different purpose based upon the individuals’ inquires and 
research questions. 

Keywords 

Research Design; Geography Education Research; Quantitative Research; Qualitative Research; Mixed 

Methods Research; Methodologies 


A Road Map for 21 st Century Geography Education: Geography Education Research 
(Bednarz, Heffron, & Huynh, 2013) addressed two questions posed as charges to the 
research committee. Based upon these two questions, the Committee issued thirteen 
recommendations to improve research in geography education. The first charge dealt 
with areas of research that would be most effective in improving geography education at 
a large scale. The recommendations include examining components and characteristics 
of exemplary geography curricula, investigating characteristics of effective geography 
teaching, and more research on fieldwork, learning progressions, and teacher 
preparation. The second charge addressed strategies and methodologies that can be 
developed and adopted to maximize the impact of research in geography education. 
These recommendations include taking interdisciplinary and/or multidisciplinary 
approaches, following established principles for scientific research, and developing and 
studying exemplary programs, curricula, tasks, measures, and assessments to build 
knowledge of effective geography teaching and learning. 

A way to respond to the recommendations of the Road Map, those particularly 
related to enhancing research effectiveness, is to systematically examine the current 
research practice of the field and find out specific areas for improvement, if any. A 
review of research methods in geography education will contribute to this by providing 
researchers with examples and guidelines to design research, select specific techniques 
for data collection and analysis, and figure out best ways to report their findings. The 
objectives of this review are to 1) analyze research studies in geography education in 
terms of research designs, specific techniques, and reporting strategies frequently 
adopted, and 2) provide examples of studies using those specific research methods as a 
future reference. Based on the findings, we also make several recommendations for 
improvement for research practice in geography education. 

Research Methods in Geography Education 

Since geography is diverse in nature, geographic research is conducted in various 
ways. Nevertheless, the research methods used could generally be categorized into one 
of the three: quantitative, qualitative, or a combination of the two, called mixed methods 
(Gomez & Jones, 2010; Clifford, Cope, Gillespie, & French, 2016). Geography 
education research is particularly concerned with the teaching and learning of 
geography. A large assortment of research designs, data collection and data analysis 
methods has been employed in an attempt to uncover how students’ knowledge and 
proficiency in geography develop and how to support the processes of learning. 

Abundant research in education has focused on how students learn in the context of 
specific disciplines and how these research findings can contribute to improving the 
curriculum and instructional practice. Therefore, education research is highly relevant 
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for geography education, and theories and research methodologies in education provide 
a model for both research and practice in geography. As Cohen, Manion, and Morrison 
(2011) point out, though, the diverse research methods used in education follow a 
similar categorization to that of geography: quantitative, qualitative, and mixed methods 

In this paper, we adopt Creswell’s (2014) categorization and definitions for these 
three major research approaches for which Creswell provides concise and well- 
developed definitions, descriptions, and explanations to refer to. It should be noted, 
however, that Creswell is far from the only individual writing about research methods, 
nor should he be the only one on which researchers rely. An overview of each research 
design, including definition and specific techniques associated, is below. 

Quantitative Research Methods 

Quantitative research is defined as “an approach for testing objective theories by 
examining the relationship among variables. These variables, in turn, can be measures, 
typically by instruments, so that numbered data can be analyzed using statistical 
procedures” (Creswell, 2014, p. 4). Researchers engaged in this form of inquiry are 
concerned with testing theories and hypotheses, controlling for alternative explanations, 
and ability to infer and generalize findings based on the sample results. Quantitative 
research encompasses a wide variety of methods, but generally the results are 
considered to be persuasive when researchers show statistical and/or practical 
significance. 

In quantitative research studies, acquiring and reporting statistical significance 
through a null hypothesis significance testing (NHST) is the most popular way to show 
how much the results are worth paying attention to. Researchers build a hypothesis, set 
a criterion for evaluating statistical significance of the results, for example, 0.05 or 0.01, 
and test the hypothesis using a NHST based on observations of samples. The p-value 
calculated from the NHST is a probability of getting the observed results from the 
samples, given that the null hypothesis is true. It is not the probability that the 
hypothesis is right or wrong. Rather, p-value is based on the assumption that the null 
hypothesis is right. For example, assume that you are interested in whether there is a 
difference in test score means between two groups of students. Your null hypothesis is 
that there is no difference in mean scores between the two groups. You conducted a 
NHST and obtained a result with a p-value of 0.03. What this tells you is that your 
observed results are only 3% likely if the null hypothesis is true. The result ends in a 
decision of whether to reject or fail to reject the null hypothesis, a dichotomous outcome 
either ‘statistically significant’ or ‘not statistically significant.’ Because 0.03 is less than 
your pre-set criterion of statistical significance (0.05), you reject the null hypothesis and 
conclude there is a statistically significant difference in mean scores between two 
groups of students. 

It should be noted that a result of NHST is considered significant not because it is 
important or meaningful, but because it has been predicted as unlikely to happen by 
chance alone (Sirkin, 2006). A p-value is an indicator to show the probability of getting 
the observed result, but it does not tell us how ‘practically’ the difference is significant 
or worth paying attention to. A concept related to importance and meaningfulness is 
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‘practical significance,’ which measures the amount of effectiveness of the sample 
results (e.g., small, medium, or large effect). Effect size (ES) is a tool for showing 
practical significance and provides quantified information about the importance of a 
research result using certain metrics (Vacha-Haase, Nilsson, Reetz, Lance, & 
Thompson, 2000). There are many effect sizes using original units, such as a mean, 
difference between two means, a percentage, and correlations, as well as using a 
standardized value, such as Cohen’s d, a regression coefficient (R 2 ), odds ratio, and eta 
square (q 2 ). Cohen’s d, q 2 , or co 2 would be more relevant to geography education 
research in relation to the quantitative methods that are frequently used in the field, such 
as t-test, ANOVA, and regression analysis (Kim, 2011). Effect sizes provide 
information to determine the importance of research results (Gravetter & Wallnau, 
2008) and are useful for replication studies to compare findings from multiple studies 
and ultimately to accumulate knowledge (Glass, 1976; Smith & Glass, 1977; Cumming, 
2006). 

Meanwhile, a confidence interval (Cl) is a type of interval estimation of a population 
parameter and is used to indicate the reliability of an estimate from samples. A Cl can 
be formed by extending two standard errors in each direction around a point estimate. 
The point estimate is the middle point of the confidence interval. Positive and negative 
allowances on either side of the point estimate provide the margins of error. The margin 
of error is half the width of a confidence interval, and it indicates the precision of the 
confidence interval (Liu, Loudermilk, & Simpson, 2014). A confidence interval can 
present statistical power of the results through its width (i.e., the shorter a Cl is, the 
stronger statistical power of the results) and offers valuable insights into the 
replicability of the study. Replicability of a study is important because replication can 
reduce the chance that the initial result was just a fluke (Cumming, 2012). Cl also 
facilitates a meta-analysis of studies by helping comparisons of multiple studies. 

NHST and values have been the dominant technique to analyze and report results 
of quantitative research in education, including geography education. However, during 
the last decade, scholars in education recognized limitations of NHST and p-values 
(Schmidt, 1996; Vacha-Haase & Thompson, 2004; Thompson, 2006; Cumming, 2012) 
and encourage researchers to report confidence intervals and effect sizes in addition 
(Wilkinson & Task Force on Statistical Inference, 1999; Cumming et al. 2007). 

Qualitative Research Methods 

Qualitative research is “an approach for exploring and understanding the meaning 
individuals or groups ascribe to a social or human problem. The process of research 
involves emerging questions and procedures, data typically collected in the particulars 
to general theses, and the researcher making interpretations of the meaning of the data” 
(Creswell, 2014, p. 4). This form of inquiry is rather inductive and focuses on individual 
meaning and importance of rendering the complexity of a situation. Qualitative research 
is based upon an “interpretative, naturalistic approach to the world” (Denzin & Lincoln, 
2013, p. 6). Qualitative research has been frowned upon because of its lack of scientific 
truths, but depending on the research questions and goals, qualitative research methods 
may be more appropriate than quantitative research methods (Denzin & Lincoln, 2013). 
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There are various methods and approaches considered qualitative in nature, and each 
provides their own significance and inquiry. Qualitative approaches allow researchers to 
investigate research questions in a way quantitative does not allow. They show depths 
and insights that can be missing from quantitative research methods and data and allow 
a voice for those who do not have one publically. Denzin and Lincoln (2013) illustrate a 
variety of qualitative research methods including, but not limited to, case studies, 
interviewing, participant observation, interpretive analysis, and visual methods. 
According to Creswell (2014) there are four basic types of collection processes: 1) 
observations, 2) interviews, 3) documents, and 4) audio and visual materials. 

First, observation requires the researcher entering the field and taking precise 
and detailed descriptions of what they observe at the research site. Observational data 
allows a researcher to gather data in an unstructured or semi-structured way about a 
particular group’s or subject’s activities, behaviors, actions, and a host of other types of 
interactions, all of which are a part of the larger human experience. The researcher can 
engage in the role of observer in several ways: complete participant, observer as 
participant, participant as observer, or complete observer. Each provides a unique 
observation experience. 

Second, interviews can be conducted face-to-face, over telephone or Skype, or as a 
focus group with multiple participants. Interviews follow a protocol to ensure the 
researcher conducts each interview the same way. Allowing open-ended questions and a 
semi-structured approach allows participants to discuss their views and opinions. Data 
gained from in-depth interviews generates direct quotes on topics, allowing for the 
opinions, feelings, and personal knowledge acquired from participants that have been 
directly involved in the particular subject or topic. 

Documents are a good source of qualitative data, leading to document analysis. This 
consists of the process of analyzing excerpts, quotations, or passages from written 
documents. These types of documents can include any type of written record 
(organizational, program, clinical), memoranda and personal or public correspondence, 
personal diaries, and responses to open-ended questionnaires and surveys (Patton, 2002; 
Creswell, 2014). Lastly, using audio and visual materials constructs data ranging from 
videotapes to sound recordings, from photographs and website main pages, to text 
messages and social media text to name a few. There are pros and cons to each 
technique, making it beneficial to collect qualitative data in a variety of forms. 

Mixed Methods Research Methods 

According to Creswell (2014), mixed methods research is conceptualized as “an 
approach to inquiry involving collecting both quantitative and qualitative data, 
integrating the two forms of data, and using distinct designs that may involve 
philosophical assumptions and the theoretical frameworks” (p. 4). The core assumption 
is that the combination of qualitative and quantitative approaches will provide a more 
complete understanding of a research problem than using either approach alone. 

Mixed methods research has been used in various disciplines, such as health and 
medicine, evaluation, educational research, sociology, and psychology (Tashakkori & 
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Creswell, 2008), illuminating the diversity of mixed methods. Since it is a relatively 
new methodological approach, researchers have a different idea of what constitutes as 
mixed methods. If data is collected qualitatively, and then analyzed quantitatively, is it 
mixed methods? What if data is collected and analyzed both quantitatively and 
qualitatively, is that mixed methods? Or, is it mixed methods if the data is collected 
both ways, but only analyzed one way? The answer to these three questions is yes. No 
matter which way the data is collected or analyzed, the research design is mixed 
methods because the data is exposed to both quantitative and qualitative methodologies. 
Since mixed methods can be executed in various ways, Creswell and Plano Clark (2007, 
2011) outlined six mixed methods designs that are commonly found in research. 
Creswell and Plano Clark provide an excellent, easy to understand explanation about 
each of these designs, so only a brief explanation of each follows. 

The first design, convergent parallel, “occurs when the researcher collects and 
analyzes both quantitative and qualitative data during the same phase of the research 
process and then merges the two sets into an overall interpretation” (Creswell & Plano 
Clark, 2011, p.77). The second, explanatory sequential design, begins with “a 
quantitative phase and follows up on specific results with a second phase. The second, 
qualitative phase is implemented [...] to explain the initial results in more depth” 
(Creswell & Plano Clark, 2011, p. 82). Exploratory sequential design, the third mixed 
methods design, is when “the researcher starts by qualitatively exploring a topic before 
building to a second, quantitative phase. [...] The primary purpose [...] is to generalize 
qualitative findings based on a few individuals from the first phase to a larger sample 
gathered during the second phase” (Creswell & Plano Clark, 2011, p. 86). The fourth 
design, embedded, “combines the collection and analysis of both quantitative and 
qualitative data within a traditional quantitative research design or qualitative research 
design. The collection and analysis of the second data set may occur before, during, 
and/or after the implementation of the data collection and analysis procedures 
traditionally associated with the larger design” (Creswell & Plano Clark, 2011, p. 90). 
The fifth is transformative design, which “occurs when researchers conduct mixed 
methods research using a theoretical-based framework, such as a transformative 
worldview. [...] The purpose of this design is to conduct research that is change 
oriented and seeks to advance social justice causes by identifying power imbalances and 
empowering individuals and/or communities (Creswell & Plano Clark, 2011, p. 96). 
Lastly, the multiphase design occurs “when an individual researcher or team of 
investigators examines a problem or topic through an iteration of connected quantitative 
and qualitative studies that are sequentially aligned, with each new approach building 
on what was learned previously to address a central program objective (Creswell & 
Plano Clark, 2011, p. 100). Each design serves a different purpose based upon the needs 
of the researcher and the research questions asked. 

Methodology 

Selection of Research Studies 

This study began by searching for empirical research articles through three academic 
geography journals that focus on the teaching and learning of geography. The three 
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journals selected were Journal of Geography, Journal of Geography in Higher 
Education, and International Research in Geographical and Environmental Education 
due to their long history of publishing articles encompassing geography education 
research. Following the objective of this study to analyze research studies in geography 
education in terms of research designs, specific techniques, and reporting strategies 
frequently adopted, we did not limit ourselves to a specific time frame, but instead took 
an exploratory approach to gather a large enough sample from each journal to review. 
We wanted to look at what type of research designs, data collection and analysis 
methods, and reporting strategies were being employed by geography education 
researchers, rather than what trends were apparent over a fixed time period (e.g. 
Bednarz, 2000). In order to get enough examples of research methodologies for our 
review we uncovered 60 research articles over a nine-year span (2005-early 2014) in the 
Journal of Geography, over a 15-year span (1999-early 2014), 58 research articles in 
the Journal of Geography in Higher Education ; and over an eight-year span (2006-early 
2014), 79 research articles were found in International Research in Geographical and 
Environmental Education. The difference in the number of research articles is due to the 
journals’ differences in scope and publication frequency. For example, International 
Research in Geographical and Environmental Education is heavy on research articles 
with a publication frequency of four times a year, while Journal of Geography is 
published six times a year and includes commentaries, book reviews, and teacher 
resource papers. A total of 197 geography education research articles were found. 
However, six articles were discarded due to lack of clarity of the research design, 
leaving 191 research articles analyzed. 

Coding 

To systematically identify and categorize specific research methods and procedures 
used in the 191 research articles, information from each article was compiled and 
organized into a database. Basic information, such as journal title, article title, key 
words, and research objectives/questions/hypotheses, were entered into the database. 
For research articles with human participants, information about sample size, age, grade 
level, gender, and geographic location, was collected. Based on the research methods 
described by the corresponding authors, information about the data collection and 
analysis procedures, statistical techniques, instruments, and other materials used, was 
also extracted from the articles and entered into the database. A summary of each article 
was also recorded. 

Using the information entered into the database, each research article was 
categorized as either quantitative, qualitative, or mixed-methods. This categorization 
was based upon the coding of each articles research method section. Codes such as 
interviews, focus groups, content analysis, open-ended questionnaires and surveys, 
journals, and observations signified qualitative data collection and analysis (Bogdan & 
Biklen, 2007; Denzin & Lincoln, 2013; Clifford, Cope, Gillespie & French, 2016). 
Whereas codes such as pre-and post-test, experiments, evaluations, Likert scales, 
validity assessment, t-tests, Pearson’s coorelation, ANOVA, and close-ended 
questionnaire and surveys were coded for quantitative data collection and analysis 
(Diggle & Chetwynd, 2011; Cumming, 2012; Clifford, Cope, Gillespie & French, 
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2016). Those articles that included both were coded as mixed methods. As noted above 
questionnaires and surveys can be considered either a quantitative or qualitative data 
collection method, based upon whether it was close-ended or open-ended. Many times 
researchers did not explicately state whether it was close-ended or open-ended. This 
leads to the importance of looking at how the data was analyzed in order to differentiate 
between a quantitative, qualitative, or mixed method research design. Therefore, close 
attention was paid to how the data was collected and how it was analyzed to ensure the 
correct catergorization of the articles. 

Each member of the research team analyzed the database and coded the articles. The 
research team crosschecked the coding results with one another to ensure each article 
was appropriately categorized. In-depth discussions took place to resolve any 
inconsistencies. Out of the 191 research articles reviewed, 55 articles were identified as 
quantitative, 65 were qualitative, and 71 coded as mixed methods. 

Results 

Quantitative Methods Used in Geography Education Research 

We analyzed 55 articles identified as quantitative research studies according to the 
research topics, statistical and other quantitative techniques used, and strategies used to 
report the study results. Over half of studies that adopted quantitative approaches (34 
out of 55) investigated effect of an educational intervention or compared effectiveness 
between different teaching strategies. Twelve (out of 55) studies investigated students’ 
and teachers’ knowledge and perceptions about various tools and pedagogical strategies 
in geography, such as Geographic Information Systems (GIS), web-based learning, and 
fieldwork. 

Forty-nine studies applied statistical analyses, and popularly used techniques include 
t-test and analysis of variance (ANOVA) to compare means and correlation coefficients 
to determine correlations among variables of interest. In six articles about developing 
assessment tools, Cronbach’s alpha was used to estimate the reliability and validity of 
the assessments. Table 1 presents frequently used quantitative methods in geography 
education and their main research objectives. 

Table 1 


Frequently used quantitative methods by research objectives 


Methods used 

Objective of 

Analysis 

Examples 

t-test and 

Comparing Variables 

Evaluating map overlay skills among 

ANOVA 


grade levels. (Battersby, Golledge, & 
Marsh, 2006) 

Factor Analysis, 

Developing and 

Developing spatial thinking ability 

Cronbach’s 

Validating 

test. (Lee & Bednarz, 2012) 

alpha, t-test, & 

Assessments 
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ANOVA 

Factor Anaylsis, 

Examing Perceptions 

Surveying constraints to GIS usage. 

t-test, & ANOVA 


(Hohnle, Schubert, & Uphues, 2013) 

Chi-square test 

Others 

Investigating the location and 
spatiality of questions in geography 
textbooks. (Jo & Bednarz, 2011) 


For the 55 studies reviewed, p-value was almost the only indicator used to determine 
significance of the results. Only eight articles reported effect, and there were four 
studies reporting a confidence interval with the results. 


Qualitative Methods Used in Geography Education Research 

A total of 65 articles that applied a qualitative research design was reviewed. Thirty- 
seven articles used interviews as the main data collection method, 12 articles used 
observation, and 40 articles used document analysis. Forty-five out of 65 articles used 
only one of the qualitative data collection methods, and the other 20 used more than 
one. All studies that used direct observation also used one or both of the other methods. 
Table 2 presents frequently used qualitative methods in geography education and their 
main research objectives. 

Table 2 

Frequently used qualitative methods by research objectives 

Methods used Objective of Analysis Examples 


Interviews Gain an Individuals’ 

Perception 


Internet-based GIS in an 
inductive learning environment: 
A case study of ninth-grade 
geography students. (Milson & 
Earle, 2008) 


Observation 


Gain an Understanding of 
the Environment and the 
Interaction Between 
Participant and Observer 


Environmental Learning Using a 
Problem-Based Approach in the 
Field: A Case Study of a Hong 
Kong School. (Kwan, 2008) 


Document Examing the Cotent of Integrating geospatial 

analysis Written/Historical/Archival technologies in an energy unit. 

Artifacts (Kulo & Bodzin, 2011) 


An example of using interviews is Milson and Earle (2008), who used a case-study 
research design to understand how to incorporate Internet-based GIS (IGIS) into a 
ninth-grade geography lesson on Africa, with the goal to understand the “implications 
of integrating IGIS into secondary geography classrooms and the potential for this 
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application [IGIS] to support inductive learning models of geography education” 
(p.227). The researchers used both classroom observations and focus-group interviews 
with students to better understand how students responded to IGIS. The interviews 
consisted of 48 students in 14 focus groups that were split into “above average,” 
“average,” and “below average” academic ability groups by the teacher. The interviews 
were semi-structured, allowing for descriptive and evaluative questions to be asked, 
along with follow-up questions for clarification on previous questions. 

Forty articles used document analysis of written documents as at least one of the 
research methods. Kulo and Bodzin (2011) used all three major qualitative research 
methods, but their use of written documents and document analysis is exemplary. The 
study focused on integrating geospatial technologies into an energy unit for 110 eighth 
graders. The researchers used content analysis of written documents as the primary 
research method, but these were supplemented with observations by the researchers, 
interviews and debriefing sessions with the teacher after each lesson, written student 
artifacts and reactions to instructional materials, assessment responses that were 
embedded in the curriculum, and student journal entries. From these materials it was 
found that students were able to improve their knowledge and understanding of energy 
sources, along with their spatial thinking skills, through the implementation of the 
lesson using geospatial technologies. 

Observation research was used by twelve articles. It was difficult to find an 
exemplary example of how observation can be used as a research method, as the articles 
that used this method did not clearly state how observation played into the findings of 
the study. In many instances, observations were written down and then analyzed as a 
written document. This was also the only research method that was not used alone for 
any of the studies. Despite the lack of replicability and research consistency across 
studies, there are exemplary models of qualitative research among the study articles for 
each method. 

Research themes and content varied greatly from article to article. Articles that used 
one or more qualitative methods covered content such as GIS (e.g., Kulo and Bodzin 
2011); fieldwork (e.g., Dummer, Cook, Parker, Barrett, & Hull, 2008); environmental 
learning (e.g., Ketlhoilwe, 2013); student conceptions of geography environmental 
learning (e.g., Bent, Bekx, & Brok, 2014); problem based learning (e.g., Kwan, 2008); 
and textbooks (e.g., Yang, 2013) in addition to many more. 

Mixed Methods Used in Geography Education Research 

Among the 71 mixed methods articles, 26 used the explanatory research design. This 
result is not surprising as many researchers are most comfortable with the collection, 
analysis, and reporting of quantitative data. Quantitative methods are also the most 
desired and accepted for journal publication. Convergent and embedded designs 
accounted for 14 articles each, exploratory design was used in nine articles, multiphase 
design was used in seven articles, and transformative was used only once. A review of 
the geography education research provided a variety of ways to carry out qualitative and 
quantitative data collection. Quantitative methods included close-ended questionnaires, 
surveys, and pre-and post-tests. Qualitative data included interviews, focus groups, 


225 




Zadrozny, ]., McClure, C., Lee, J., & Jo, I. (2016) / Designs, Techniques, and Reporting Strategies ... 

open-ended questionnaires, and observations. There were articles that collected data 
qualitatively, but analyzed it quantitatively, and vice versa, simply showing the vast 
ways to coduct research. 

An explanatory mixed methods design was applied by Wang and Chen (2013), where 
the researchers used a questionnaire and personal interviews to investigate Taiwanese 
teacher perceptions of GIS teaching, the popular pedagogies used by GIS teachers, the 
constraints that hinder the implementation of GIS teaching, and the distinguishing 
characteristics of GIS education in Taiwan. They explicitly state in their article that the 
personal interviews were conducted to supplement the results of the questionnaire and 
to obtain further information, identifying the key components of an explanatory design. 
The questionnaire included six categories, which were analyzed using Kolmogorov- 
Smirnov test, Shapiro-Wilk test, and Wilcoxon rank-sum test. 

An example of a convergent mixed methods design is DiBiase and Kidwai’s (2010) 
study. The research questions they posed were (1) why do older students seem to thrive 
better in this introductory course in geographic information, science, and technology, 
even though younger students are allegedly more accustomed to computing and 
information technologies and (2) what differences in student behavior, course design, 
and instructor roles should faculty members and administrators anticipate in relation to 
the ages of the students they plan to engage online? In order to answer these questions, 
they performed a quasi-experimental study and employed a mixed methods research 
design including quantitative and qualitative data. The experiment compared students 
from an undergraduate course (GEOG 121) and those from a continuing adult 
professionals course (GEOG 482). The data was collected and analyzed in several ways 
including records of student activity within the learning management system, student 
emails and discussion board postings, activity diaries kept by the lead author, surveys of 
student attitudes, and semi-structured telephone interviews with select undergraduate 
students. The quantitative and qualitative data was collected simultaneously throughout 
the course, and it was analyzed together to understand and explain the differences in 
performance and attitudes of younger and older adult learners. 

An embedded mixed methods design can be found in Huang’s (2011) article. Huang 
carried out an experimental design by developing two websites providing a different 
learning environment, one structured and one exploratory, which was used by two 
groups of students. As the students explored the web sites, the screens were videotaped 
to analyze students’ Web-browsing behavior. Other quantitative and qualitative data 
was collected in the form of a geography achievement test on the content of the web 
sites, a student worksheet to be answered during the browsing of the web sites, and 
interviews and observations. The quantitative data were analyzed by using t-tests, and 
the interviews and observations helped the researcher understand students’ cognitive 
model. The qualitative data was collected and analyzed to get a better understanding of 
the student actions. The qualitative data embedded in the primary quantitative research 
design provided a deeper understanding. 

Solem and Foote’s (2006) article is an example of exploratory mixed methods 
design, exploring the experiences of early career geography faculty by using focus 
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group interviews as a source of qualitative data, which helped formulate hypothesis for 
a more detailed study using a written questionnaire. The interviews, analyzed using 
HyperRESEARCH software, identified fifteen distinct career issues and perceived 
effective strategies for coping with these issues. In order to verify the significance of the 
career issues raised in the focus groups and their relationship with social and 
professional characteristics, a written survey was administered to a sample of faculty to 
collect quantitative data. Factor analysis, Principal Components Analysis (PCA) with 
Varimax rotation, Cronbach’s alpha, Mann-Whitney U test, and the Kruskal Wallis test 
were all ways the quantitative data were analyzed. 

An article by Boehm, Brysch, Mohan, & Backler (2012) provides an example of a 
multiphase mixed methods design. An online professional development series targeted 
for pre- and in-service grade 6-9 geography, science, and environmental science 
teachers was proposed as a new way to provide professional development for geography 
teachers. In order to determine the receptiveness of an online program compared to 
face-to-face, there were multiple phases of data collection. During this first phase of 
preliminary research, four groups were surveyed, each with a different set of open and 
close ended questions, to get an idea of the potential value and use of the program. The 
second phase involved an experiment that compared the online professional 
development program to traditional face-to-face professional development with two 
groups of teachers. After each group experience both styles of professional 
development, they were asked to complete a Likert-scale survey to compare the two 
styles, which was analyzed using a chi-square goodness-of-fit test. 

Lastly, the article by Brinegar (2001) shows an example of a transformative mixed 
methods design. The paper is written from a feminist perspective, and the overall 
purpose is to report on women’s progress in the discipline of geography by using a 
variety of data sources to analyze. The data sources used included figures from the 
National Center for Education Statistics and the National Science Foundation, journals 
and publications produced by the Association of American Geographers, and a 
qualitative survey from US doctorate-granted geography departments. Female 
representation both in faculty and students was determined by the percentages and 
number graduated. The percentages of research articles authored by women was also 
documented and analyzed. 

Discussion 

Out of the 191 research articles reviewed, 55 articles were identified as quantitative, 
65 qualitative, and 71 were identified as mixed methods studies. As research design is 
closely related to specific types of inquiry and research questions, the variety of 
research designs found can be considered a reflection of the numerous inquiries and 
research questions investigated in geography education. Despite the relatively fewer 
numbers of quantitative studies it does not mean that geography education researchers 
were less concerned with research questions that can be investigated through 
quantitative research techniques. Instead, it could be attributed to the publishing 
tendency of quantitative results. According to Cummings (2012), researchers whom fail 
to get a statistical significant finding are hesitant to submit their work for publication, 
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while those who have statistical significance are more likely to be accepted and 
published This is also called the “file drawer problem” as suggested by Rosenthal 
(1979), where publication bias occurs in journals by depicting only 5% of studies 
conducted while the other 95% are filed away due to nonsignificant results. In contrast, 
research that adopted qualitative or mixed methods research designs is relatively free 
from the restraint of statistical significance. 

This study found that the most popular statistical analysis methods in geography 
education are t-test and the analysis of variance (ANOVA) as most of the studies 
involve comparing two or more means. For reporting the results of analysis, p-valuc 
was almost the only indicator used to determine significance of the findings. Few 
studies reported confidence intervals or effect sizes, and no single study reported both 
together. According to the American Psychological Association (2009), reporting both 
confidence intervals and effect sizes has been highly recommended in reporting 
quantitative research in education. Scholars also suggest that effect sizes can be more 
useful when reported with confidence intervals (Preacher & Kelly, 2011). In addition, 
according to the recommendations by American Psychological Association for reporting 
educational research, all basic information about the data, such as descriptive statistics 
(e.g., mean, standard deviation, and original units) must be reported at all times because 
providing such information helps replicating the study and eventually accumulating 
knowledge. However, reporting confidence intervals and effect sizes is still uncommon 
in geography education research. 

There is little consistency in the types of studies undertaken using qualitative 
methods. While the methods used for this study were placed into three main categories, 
the sub-categories varied widely. The 65 articles that were categorized as qualitative 
adopted a total of 26 slightly different methods to collect data. Among the studies 
reviewed, there was not a single pair of articles of which the results are comparable. 
Each study deals with perceptions, opinions, understandings, and other not-easily 
measured characteristics of a specific group of individuals in a unique context. 

As mentioned earlier, data can be collected and analyzed in various combinations of 
quantitative and qualitative methods to be considered mixed methods. This being said, 
there is not one correct way to perform mixed methods research. Each article identified 
the data collected, a mandatory section of a published research article, but very few of 
the articles discussed the data collection methods and analysis exclusively or 
extensively. This made it difficult to decipher which mixed methods design was used 
based upon Creswell and Plano Clark’s design definitions (2007, 2011), leaving it up to 
the judgement of the research team. 

The purpose of this review was to examine research methods, techniques, and 
reporting strategies that have been commonly used in geography education and provide 
a list of example research studies representing those methods. This article does not 
discuss in detail all of the various ways data is collected and analyzed but we do hope it 
provides a starting point on what type of research design and methodological approach 
you will want to conduct with some examples of how it was used in context. Some 
researchers may want to explore large statistical datasets, such as the National 
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Assessment of Educational Progress (NAEP) results, while others want a more personal 
experience concentrated on interviews and focus groups to understand a problem. 
Whichever type of researcher you may be it is important to know where to begin. This 
paper will serve geography education researchers, especially for early career scholars 
and graduate students, as a reliable reference to reviewing and comparing various 
research methods used in the field and deciding appropriate ones for their own research 
endeavor. 


Conclusions and Recommendations 

A Road Map for 21 st Century Geography Education: Geography Education Research 
Committee called for an increase in replicable studies that will increase the knowledge 
and quality of research in geography education. This is another call to the geography 
education research community to increase the number of sound research protocols and 
designs in the discipline (Downs 1994, Stoltman, 1997). In order to build a base for 
future research that is grounded on the previous research design and findings, a clear 
and detailed report of the methodologies and the results is critical. The views expressed 
in this article are based upon the authors and we do not expect everyone to agree. We 
acknowledge that our findings may be different than the original intentions of the 
authors cited and studied. However, our conclusions express our recommendation for an 
immediate call of action to publish stronger research articles in geography that build 
upon existing research literature and studies, producing more longitudinal and 
transformative works. 

We looked at quantitative studies and saw the majority of researchers use NHST, and 
we discuss the advantages of including effect sizes and confidence intervals into the 
results to provide a more accurate picture of the research. It is not the purpose of this 
article to argue that we should follow this new method because other disciplines have 
already accepted it. Rather, we suggest that reporting practical significance of the study 
findings contribute to building a more comprehensive knowledge about the phenomena 
investigated in geography education research studies. 

Based on the qualitative studies reviewed, we conclude that geography education 
researchers can benefit from more coordinated research efforts across topic. The 
primary purpose of these studies was to better understand how various teaching styles, 
project types, learning environments, and other factors influence how geography is 
taught and learned across age groups, educational levels, and cultural backgrounds. It 
would be worthwhile to investigate the same research questions using the same 
methodologies but with different participants in different geographical and educational 
settings. This can be accomplished when studies report the methodologies and research 
procedures in greater detail. 

A mixed methods methodology is a valuable research design in the field of 
geography education. Collecting and analyzing both quantitative and qualitative data 
proves to be beneficial in improving various aspects of research. Whether it is about 
GIS, professional development, or the teaching and learning of geography, mixed 
methods can help answer the research questions in a unique way different from a lone 
quantitative or qualitative research design. We suggest that when submitting a mixed 
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methods study for publication, the article should address all the necessary components 
of a mixed methods design as outline by Creswell and Plano Clark (2007, 2011). These 
components include: data collection methods; data analysis methods; the timing of 
when the methods were employed, such as qualitative then quantitative, vice versa, or at 
the same time; and, the purpose of conducting mixed methods. Following this reporting 
style is a good habit no matter what methodology and design is used. 

We strongly encourage future geography education researchers to look at how 
previous research in geography education has been done and take this article into 
consideration when developing their own research designs. Research can be 
intimidating, but there are plenty of exemplar studies out there that we can refer to 
guiding our research questions. The geography education community must find a way to 
move forward effectively so that in ten years we are not repeating ourselves about the 
need for more data and research. 
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